Pixel sampling for spatial light modulator

ABSTRACT

An image forming system includes a spatial light modulator (SLM) including a plurality of pixels. Each pixel is configured to diffract incident light and cause the diffracted light to exit the SLM, where a first diffraction order of light exiting the SLM passes through a first exit pupil and higher diffraction orders of light exiting the SLM pass through additional exit pupils having different positions from the first exit pupil. Control logic operatively coupled to the plurality of pixels is configured to control each pixel to control its modulation of the light incident on the pixel and cause the plurality of pixels to collectively form an image at each exit pupil. A light source is configured to emit incident light toward the SLM. A resampling layer is configured to subsample each pixel electrode with two or more samples per pixel to increase a spacing between each exit pupil.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.16/355,189, filed Mar. 15, 2019, the entirety of which is herebyincorporated herein by reference for all purposes.

BACKGROUND

An optical system may include an image forming optic illuminated by anillumination source to produce a viewable image. Image forming opticsmay be transmissive, such that an image is formed by modulating lightpassing through the image-forming optic, or reflective, such that animage is formed by modulating light reflected from the image-formingoptic.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example hologram and image output representation.

FIGS. 2A, 2B, and 2C schematically show an image forming systemincluding a spatial light modulator (SLM).

FIG. 3 schematically illustrate presentation of virtual imagery using ahead-mounted display device (HMD).

FIGS. 4A and 4B illustrate sampling of pixels of a SLM.

FIGS. 5A and 5B schematically illustrate spacing of exit pupils formedby an image forming system.

FIG. 6 schematically shows an example image forming system with a lightsource configured to emit incident light at a dynamically controllableincidence angle.

FIG. 7 schematically shows an example image forming system with twolight sources.

FIG. 8 schematically shows an example computing system.

DETAILED DESCRIPTION

Phase modulating devices, such as spatial light modulators (SLMs), mayform an intensity distribution using diffraction. The intensitydistribution may be an image that is used as an information display, forprojecting an array of spots for depth sensing, used for illuminationpurposes in vehicles and buildings, etc. The phase modulating device canbe either transmissive (like many large panel liquid crystal displays(LCDs)) or reflective, like Liquid Crystal over Silicon (LCoS) devices.

SLMs are often pixelated devices with square pixels placed on a regulargrid. A spatially quantized device, with rows and columns of pixels,will typically form a central diffraction pattern and repetitions ofthis pattern in the horizontal and vertical axes. These repetitionsoriginate from higher diffraction orders of light. For instance, a firstdiffraction order may represent an intended image, while higherdiffraction orders of light may cause repetitions of the image to beformed at other positions. These higher diffraction orders are formedbecause of aliasing; any regularly sampled signal will have its spectrumrepeated at a frequency equal to the sampling frequency.

Higher diffraction orders of light are often considered to beundesirable. For instance, such higher orders may form images that areoverly close to a central image, causing both the central image and oneor more repetitions of the image to enter a user eye at once. Forexample, FIG. 1 schematically shows a representation of an aliased image102 produced by a hologram 104. As illustrated, the central, or desired,image 106 (e.g., the box with the letter “F” in the middle) is repeatedas diffraction orders (e.g., as indicated by the “X” through theduplicated boxes) along horizontal and vertical axes of the image 102(e.g., above, below, and to the side of the central, desired image).These repetitions of the central spectrum/signal are referred to ashigher orders. The higher orders may be on the far field of the SLM(e.g., Fourier plane) or an intermediate plane. The central or thedesired image may lay on the far field plane and/or an intermediateplane.

The higher orders would have the same intensity as the central image ifthe pixels had an infinitesimal size; e.g., if pixels were effectivelypoint sources. However, as pixel size is finite, the diffraction patternreduces in amplitude further away from the central order. This is calledaperture theorem and in the specific case of Fraunhofer/Fourierholograms with square pixels, the image is multiplied by a sincfunction, usually referred to as the sinc envelope. The higher ordersare usually unwanted side effect of a diffractive system, and thusvarious techniques are often used to eliminate the unwanted orders. Thiscan remove undesirable visual artifacts related to order spacing, aswell as increase the power available for the central, desired order.However, such techniques often increase the bulk and/or complexity ofthe overall device.

Furthermore, it is a common goal in the design of image forming systems,such as holographic display devices, to increase the size of an “eyebox”in which a human eye pupil can receive image light from the system,especially when a large field-of-view (FOV) is desired. Image lightcreated by such image forming systems is typically focused toward anexit pupil. If the exit pupil coincides with a human eye pupil then thehuman may perceive the intended image. Thus, expanding the eyebox may beachieved by creating more exit pupils and/or increasing the ability ofthe system to move an exit pupil to track a moving eye.

Accordingly, the present disclosure is directed to techniques for usingthe higher diffraction orders of light that are typically undesirable oreliminated. These higher orders are used to form additional exit pupilsand thus increase the size of the eyebox. To this end, the image formingsystem includes a SLM having a resampling layer that samples each pixelof the SLM with two or more samples per pixel. This affects the spacingat which the image is repeated via the higher orders of light. As aresult, multiple exit pupils may be formed having a sufficient spacingsuch that only one exit pupil coincides with a human eye pupil at atime, mitigating the undesirable visual artifacts that are commonlyassociated with the higher diffraction orders. Furthermore, the pixelsare sampled such that each sample resembles a point source, thusincreasing the brightness of the resulting higher diffraction orders.Because the samples still have a finite size and are not truly pointsources, some amount of intensity loss may still be observed among thehigher diffraction orders. However, given the small size of the samples,this amplitude loss may be mitigated such that the higher diffractionorders form sufficiently clear and bright images at respective exitpupils.

FIG. 2A schematically shows portions of an example image forming system200 including an SLM 202. SLM 202 may take any suitable form and use anysuitable technology. Though the present disclosure primarily focuses onreflective SLMs (e.g., such as LCoS implementations), transmissive SLMsmay instead be used. It will be understood that FIG. 2A shows theexample image forming system 200 schematically and is not drawn toscale. FIGS. 2B, 3, 4A, 4B, 5B, and 6-8 are similarly schematic innature.

SLM 202 includes a plurality of pixel electrodes, including pixelelectrodes 204A, 204B, and 204C. Each pixel electrode is configured toreflect light incident on the pixel electrode to exit the SLM. Forinstance, FIG. 2B shows a single beam of incident light reflecting frompixel electrode 204B. The reflected light includes a first order lightbeam shown as a solid line as well as two higher order light beams shownin dashed lines. As will be described in more detail below, each ofthese light beams may exit the SLM to pass through respective exitpupils. For instance, the first diffraction order of light may passthrough a first exit pupil, while higher diffraction orders of light maypass through one or more additional exit pupils having differentpositions from the first exit pupil.

The plurality of pixel electrodes, in tandem with a phase modulatinglayer 206 of the SLM, may collectively comprise a plurality ofindividual pixels 205A-205C configured to diffract incident light andcause the diffracted light to exit the SLM. For instance, the pluralityof pixels may be operatively coupled with control logic (e.g., viaattachment to corresponding pixel electrodes), the control logic beingusable to influence electrical conditions at each pixel electrode. Thismay in turn affect electrical conditions within the phase modulatinglayer (e.g., liquid crystal layer) 206 of the SLM. The phase modulatinglayer affects the phase of any incident/reflected light passing throughit, thereby diffracting the light into various diffraction orders. Thus,by altering electrical conditions within the phase modulating layer viathe pixel electrodes, the phase of the incident/reflected light may bechanged. In this manner, the control logic may be used to control eachpixel to control its modulation of the light incident on the pixel andcause the plurality of pixels to collectively form an image at each exitpupil. Depending on the implementation, each pixel of the SLM maycorrespond to a single pixel in the resulting image, or each pixel inthe SLM may contribute to a plurality of pixels (e.g., some or all thepixels) in the resulting image.

Though not shown in FIG. 2A, the SLM may in some cases include anadditional electrode, such as a ground electrode or common electrode,that may be disposed on the opposite side of the phase modulating layerfrom the plurality of pixel electrodes.

SLM 202 may also include a blurring layer 208 disposed between the pixelelectrodes and phase modulating layer. Imperfections in a phasemodulating layer, such as a liquid crystal layer, may result inundesirable visual artifacts in a formed image. For instance, a phase ofthe phase modulating layer may not change uniformly or optimally betweenpixels, causing aberrations to occur when the pixels are sampled neartheir edges. This can be mitigated through use of a blurring orinterpolation layer, which may have a higher permittivity than the phasemodulating layer. In this manner, the blurring layer may be configuredto smooth phase transitions of a liquid crystal state of the phasemodulating layer between localized areas associated with the pixelelectrodes/pixels. The blurring layer may have any suitable arrangementand be constructed from any suitable materials. In one example, theblurring layer may be constructed from lead zirconate titanate (PZT).More details regarding the blurring layer are described in U.S. patentapplication Ser. No. 15/257,581, the entire contents of which are herebyincorporated herein by reference for all purposes.

FIG. 2B schematically shows another view of image forming system 200.System 200 is operable to position an exit pupil and provide an eyeboxin which virtual imagery generated by the system is viewable. System 200may be implemented in any suitable display device. For instance, theimage forming system may be a component of a head-mounted display device(HMD) including a near-eye display (NED). Thus, image forming system 200may in some cases correspond to a single eye and be paired within a NEDand/or HMD with a second, similar display system for displaying imageryto a second eye. In other scenarios, however, the image forming systemmay be a component of other suitable display devices, such as mobilephones, television displays, computer monitors, etc.

As discussed above, image forming system 200 includes an SLM 202, whichmay take any suitable form. Image forming system 200 also includes alight source 210 configured to emit incident light (Li) toward the SLM.In some cases, and as will be discussed in more detail below withrespect to FIG. 6, the light source may be configured to emit incidentlight toward the SLM at a dynamically controllable incidence angle. Thismay be used, for instance, to steer the position at which an exit pupilforms, thereby increasing the size of the eyebox. For instance,dynamically controlling the incidence angle may change the respectivepositions at which each exit pupil forms—e.g., an exit pupilcorresponding to a first order of light as well as additional exitpupils corresponding to higher orders of light. As one example, thelight source may include a microprojector and a steerable micromirror,although any suitable light-producing devices and optics may be used.The light source may output light having any suitable wavelength orspectrum, and in some cases may output collimated light.

In the depicted example, light emitted from light source 210 enters SLM202. The input angle at which light is introduced into the SLM may becontrollable in various manners. As one example, the light source mayinclude a steerable micromirror controllable to change the angle atwhich light from a microprojector is introduced into SLM 202 to steer anexit pupil of the image forming system 200. In other examples, and aswill be described in more detail below with respect to FIG. 7, differentlight sources arranged at different angles may be used to vary an inputangle by selecting which light source to use for illumination at anygiven time, and/or any other suitable method of varying a light inputangle may be used.

While not depicted in FIG. 2B any suitable elements may be disposedbetween the light source and SLM. For instance, light from the lightsource may first pass through a waveguide having suitable in-couplingand out-coupling elements. Furthermore, although FIG. 2B only shows asingle beam of light entering the SLM, this is only for the sake ofvisual clarity. In practical implementations, any suitable portion ofthe SLM may be illuminated by incident light from the light source,including the entire SLM.

For the light input angle illustrated in FIG. 2B, a portion of the lightexiting the SLM, corresponding to a first diffraction order of light(Lr1) converges toward an exit pupil 212 proximate a human eye pupil214. Light passing through the exit pupil may therefore enter the eyevia human eye pupil 214 and strike the retina, causing the light to beperceived as an image. Though FIG. 2B depicts the light focusing at apoint outside of the human eye pupil, this is for illustration purposesonly. In practical use, the light may converge toward a focal point thatlies before, within, or beyond the human eye. In some examples, the exitpupil formed may coincide with the human eye pupil. Light entering thehuman eye pupil may be focused by the eye lens to modify the light'sfocal point, for example to focus the light at the retina.

FIG. 2B also shows another exit pupil 212B formed by light representinga higher diffraction order (Lr2). In the illustrated example, both exitpupils 212A and 212B are proximate the human eye pupil. Thus, lightpassing through both exit pupils may enter the human eye pupil and beperceived as a doubled image. However, as will be discussed in moredetail below, the spacing between exit pupils corresponding to differentdiffraction orders of light may be increased by subsampling pixels ofthe SLM.

Furthermore, in the illustrated example, light exiting the SLM isfocused toward respective focal points. To this end, the image formingsystem may include any suitable focal optics in the light path betweenthe SLM and human eye. For instance, the light may pass through anystatic lenses, dynamic lenses, holograms (e.g., thin or thick Braggholograms), waveguides, etc., en route to the human eye. In this manner,the SLM need not be positioned directly in front of the eye as is shown,but rather may have another suitable position, and light exiting the SLMmay take any suitable path and pass through any suitable optics beforereaching the exit pupil.

As discussed above, in some cases the image forming system may beincluded as part of an HMD or NED, in which paired images are presentedto two eyes at once. When the two images are stereoscopically offset,the virtual imagery may be perceived as a three-dimensional object thatappears to exist at a three-dimensional position within the wearer'senvironment. This is shown in FIG. 3, which shows an overhead view of awearer using a NED. As shown, the NED includes two image forming systemsincluding a first system 300L, positioned in front of the wearer's lefteye pupil 302L, and a second image forming system 300R positioned infront of right eye pupil 302R. It will be understood that image formingsystem 200 shown in FIGS. 2A and 2B may be either of systems 300L and300R. Similarly, human eye pupil 214 shown in FIG. 2B may be either ofpupil 302L and 302R shown in FIG. 3A. A virtual image 340 is visible tothe wearer as a virtual object that appears to be present at athree-dimensional position some distance away from the wearer. Asdiscussed above, such an image may be formed via light exiting an SLMtoward exit pupils formed by the image forming systems and in turnentering human eye pupils 300L and 300R.

Returning to FIG. 2B, in some examples, image forming system 200 mayvary the exit pupil location based on the location of the user eyepupil. Thus, FIG. 2B shows an eye-tracking system 216 configured totrack a current position of the human eye pupil. In some examples,eye-tracking system 216 may include a light source that projects lightonto the eye, and an image sensor that captures light reflected from thecornea, or other structures within the eye, with which glints and/orother features can be identified to determine the pupil location. Thepupil location identified by eye-tracking system 216 may be provided tocontrol logic 218, which may be configured to set a current angle of theincident light emitted by the light source, thereby causing an exitpupil to form proximate to the current position of the human eye pupil.

Furthermore, control logic 218 may be operatively coupled to theplurality of pixels via coupling with the pixel electrodes of the SLM.By altering electrical conditions at each pixel electrode, the controllogic may influence light modulation occurring at each pixel. In thismanner, the plurality of pixels can be made to collectively form animage at each exit pupil (e.g., each exit pupil corresponding todifferent diffraction orders of light). Controller 218 may beimplemented as any suitable processing componentry, including logicmachine 802 described below with respect to FIG. 8.

As noted above, in FIG. 2B two exit pupils 212A and 212B are proximateto the same human eye pupil 214. This is also shown in FIG. 2C, whichshows an example eyebox 220 in which human eye pupil can receive lightfrom the image forming system 200. For the sake of simplicity, as usedherein, an “eyebox” will typically refer to a two-dimensionalrectangular region of space, although in practical applications aneyebox may have any suitable shape and dimensions.

In FIG. 2C, both of exit pupils 212A and 212B are proximate to human eyepupil 214. As such, light passing through both exit pupils may bereceived by the human eye at the same time. Thus, the user may perceivetwo copies of the same image corresponding to different diffractionorders of light.

To address this, as discussed above, each pixel of the SLM may besubsampled with two or more discrete samples per pixel. This isschematically illustrated in FIG. 4A, which shows portions of anotherexample image forming system 400 including an SLM 402. As with SLM 202,SLM 402 includes a plurality of pixel electrodes 404A-404C, a phasemodulating layer 406, and a blurring layer 408. The pixel electrodes andphase modulating layer collectively comprise a plurality of pixels405A-405C configured to diffract and modulate incident light to form animage. However, SLM 402 also includes a resampling layer 410 disposedbetween the light source and phase modulating layer. The resamplinglayer interacts with the incident light such that each pixel of theplurality is selectively sampled at two or more specific regions.Focusing specifically on pixel electrode 404B, a first light beam isreflected as a first sample 412A, while a second light beam is reflectedfrom the same pixel electrode as a second sample 412B.

FIG. 4B shows the plurality of pixels from another angle. Specifically,FIG. 4B shows each of pixel electrodes 404A, 404B, and 404C arranged ina regular grid along with a plurality of other pixels. Samplinglocations on each pixel, including samples 412A and 412B, are indicatedwith black circles. As shown, each pixel is sampled four times, althoughthis is not limiting. Rather, any suitable number of samples may be usedper pixel depending on a desired spacing between the various diffractionorders of light exiting the SLM.

The specific positions of each sampling location on the various pixelsare also not limiting. As discussed above, imperfections in a phasemodulation layer, such as a liquid crystal layer, may cause visualaberrations when pixels are sampled near their edges. Thus, it may insome cases be desirable to sample a central region of each pixel.However, such visual aberrations may be mitigated through use of ablurring layer in the SLM, potentially allowing edge regions of eachpixel to be sampled without introducing aberrations.

The spacing between each pixel sample may be set based on a size of eachpixel as well as a desired exit pupil spacing. In the illustratedexample, the intra-pixel spacing between each of the two or more samplesper pixel is substantially the same for each of the plurality of pixels.Furthermore, the two or more samples for each pixel collectively form aregular sample grid. In other words, a spacing between two adjacentsamples is substantially the same regardless of whether the two adjacentpixels are on a same pixel or adjacent pixels. However, this is notlimiting. In other examples, any suitable spacing and sample geometrymay be used, including irregular spacings.

The resampling layer used to sample each pixel may take any suitableform. In one example, the resampling layer may take the form of a maskthat selectively blocks incident light from reaching at least a portionof each of the plurality of pixels. Alternatively, the resampling layermay take the form of an optical array element, such as a microlens arraythat focuses incident light at specific portions of each pixel. This mayensure that more of the incident light is used, rather than beingblocked by a mask. As another example, the resampling layer may take theform of a diffractive optical element, such as a holographic opticalelement (HOE)—e.g., a thick or thin Bragg hologram. Additionally, oralternatively, the resampling layer may be configured to modify thegeometric phase of the incident light, such as a metamaterial. When adynamic hologram or geometric phase metamaterial is used, the samplespacing on the pixels may be dynamically changed. In other words, theper-pixel sample spacing or sample quantity with which each pixel issubsampled by the resampling layer may be dynamically controllable—e.g.,to affect the angles at which each order of diffracted light exits theSLM. Thus, the spacing between each exit pupil formed by the variousdiffraction orders of light may be changed dynamically, for instance tofollow a moving eye pupil.

In one example, the resampling layer may take the form of a surfacerelief grating (SRG) or out-coupling layer on a waveguide configured totransmit light via total internal reflection. If the outcoupling fromthe waveguide is relatively weak, light reflecting back from the SLM maybe weakly affected by the outcoupling structure en route to the eye.This may be addressed by using an outcoupler that is polarizationdependent. For instance, in addition to affecting the wavelength ofincident light, the SLM may affect the polarity of light it reflects,such that the light is no longer affected by the waveguide outcoupler.Furthermore, when an outcoupling waveguide is used, the outcoupler maybe designed such that the resampling for red, green, and bluewavelengths is different and inversely proportional to wavelength, suchthat order spacing is substantially identical for all three colors.

Returning briefly to FIG. 4A, only one diffraction order of light (i.e.,the first order) is shown reflecting from each pixel electrode. However,this is only for the sake of visual clarity. Rather, as discussed above,any regularly sampled signal will have its spectrum repeated in a mannerthat is dependent on the sampling frequency. Accordingly, increasing thesampling frequency of the grid of pixels changes the spacing between thevarious diffraction orders of light exiting the SLM. In practice, thisincreases the spacing between exit pupils that the various diffractionorders of light pass through. In this manner, the size of the eyebox caneffectively be expanded, as the various diffraction orders of light canbe used to form additional exit pupils with sufficient spacing that thehuman eye pupil only ever receives light passing through one of the exitpupils.

This is illustrated in FIGS. 5A and 5B. FIG. 5A schematically showsanother example image forming system 500 including an SLM 502. Incidentlight (Li) emitted by a light source 504 is diffracted by pixels of theSLM and passes through exit pupils 506A and 506B. Specifically, a firstdiffraction order of light passes through exit pupil 506A, which isproximate to a human eye pupil 508, while a higher diffraction order oflight passes through exit pupil 506B, which is not proximate to thehuman eye pupil.

This is also shown in FIG. 5B, which shows an eyebox 514. In FIG. 5B,first exit pupil 506A is proximate to human eye pupil 508, while secondexit pupil 506B is not. In other words, the spacing between each exitpupil at which the image is formed is greater than the human eye pupilwidth. The system may in some cases set this spacing to be greater thana predetermined human eye pupil width—for instance, an eye pupil widthcorresponding to an average eye pupil size, a 99^(th) percentile eyepupil size under ordinary daylight conditions, a specific eye pupilwidth corresponding to a known user, a measured eye pupil width for acurrent user, or another suitable spacing may be used.

As indicated above, in some scenarios, various techniques may be used tofurther expand the eyebox by moving the positions at which each exitpupil is formed. For instance, FIG. 6 again shows image forming system500 of FIG. 5A. However, in FIG. 6, the angle of incidence of lightentering the SLM has been changed. In other words, light source 504 isconfigured to emit the incident light at a dynamically controllableangle. As a result, the angles at which all light exits the SLM,including all diffraction orders of light, has changed. This in turnchanges the positions of each respective exit pupil that the variousdiffraction orders of light pass through, as exit pupils 506A and 506Bare at different positions from what is shown in FIG. 5A. This may bedone, for example, to account for a changed position of human eye pupil508 for instance based on feedback provided by eye-tracking system 510.

Additionally, or alternatively, an image forming system may include morethan one light source. For instance, FIG. 7 schematically shows anotherexample image forming system 700 including an SLM 702, light source704A, eye-tracking system 710, and control logic 712. Light exiting theSLM forms exit pupils 706A and 706B on its way toward human eye pupil708. However, image forming system 700 also includes a second lightsource 704B configured to emit incident light toward the SLM at adifferent incident angle from first light source 704A. As a result, eachof the first exit pupil (corresponding to the first diffraction order oflight) and one or more additional exit pupils (corresponding to one ormore higher diffraction orders of light) may be formed at respectivefirst positions when the first light source emits incident light towardthe SLM. By contrast, each exit pupil may be formed at respective secondpositions when the second light source emits incident light toward theSLM. In this manner, the positions at which each of the exit pupils formmay be switched between two or more discrete positions by switchingwhich light source is active, for instance based on feedback fromeye-tracking system 710.

In some embodiments, the methods and processes described herein may betied to a computing system of one or more computing devices. Inparticular, such methods and processes may be implemented as acomputer-application program or service, an application-programminginterface (API), a library, and/or other computer-program product.

FIG. 8 schematically shows a non-limiting embodiment of a computingsystem 800 that can enact one or more of the methods and processesdescribed above. Computing system 800 is shown in simplified form.Computing system 800 may take the form of one or more personalcomputers, server computers, tablet computers, home-entertainmentcomputers, network computing devices, gaming devices, mobile computingdevices, mobile communication devices (e.g., smart phone), and/or othercomputing devices.

Computing system 800 includes a logic machine 802 and a storage machine804. Computing system 800 may optionally include a display subsystem806, input subsystem 808, communication subsystem 810, and/or othercomponents not shown in FIG. 8.

Logic machine 802 includes one or more physical devices configured toexecute instructions. For example, the logic machine may be configuredto execute instructions that are part of one or more applications,services, programs, routines, libraries, objects, components, datastructures, or other logical constructs. Such instructions may beimplemented to perform a task, implement a data type, transform thestate of one or more components, achieve a technical effect, orotherwise arrive at a desired result.

The logic machine may include one or more processors configured toexecute software instructions. Additionally, or alternatively, the logicmachine may include one or more hardware or firmware logic machinesconfigured to execute hardware or firmware instructions. Processors ofthe logic machine may be single-core or multi-core, and the instructionsexecuted thereon may be configured for sequential, parallel, and/ordistributed processing. Individual components of the logic machineoptionally may be distributed among two or more separate devices, whichmay be remotely located and/or configured for coordinated processing.Aspects of the logic machine may be virtualized and executed by remotelyaccessible, networked computing devices configured in a cloud-computingconfiguration.

Storage machine 804 includes one or more physical devices configured tohold instructions executable by the logic machine to implement themethods and processes described herein. When such methods and processesare implemented, the state of storage machine 804 may betransformed—e.g., to hold different data.

Storage machine 804 may include removable and/or built-in devices.Storage machine 804 may include optical memory (e.g., CD, DVD, HD-DVD,Blu-Ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM,etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive,tape drive, MRAM, etc.), among others. Storage machine 804 may includevolatile, nonvolatile, dynamic, static, read/write, read-only,random-access, sequential-access, location-addressable,file-addressable, and/or content-addressable devices.

It will be appreciated that storage machine 804 includes one or morephysical devices. However, aspects of the instructions described hereinalternatively may be propagated by a communication medium (e.g., anelectromagnetic signal, an optical signal, etc.) that is not held by aphysical device for a finite duration.

Aspects of logic machine 802 and storage machine 804 may be integratedtogether into one or more hardware-logic components. Such hardware-logiccomponents may include field-programmable gate arrays (FPGAs), program-and application-specific integrated circuits (PASIC/ASICs), program- andapplication-specific standard products (PSSP/ASSPs), system-on-a-chip(SOC), and complex programmable logic devices (CPLDs), for example.

The terms “module,” “program,” and “engine” may be used to describe anaspect of computing system 800 implemented to perform a particularfunction. In some cases, a module, program, or engine may beinstantiated via logic machine 802 executing instructions held bystorage machine 804. It will be understood that different modules,programs, and/or engines may be instantiated from the same application,service, code block, object, library, routine, API, function, etc.Likewise, the same module, program, and/or engine may be instantiated bydifferent applications, services, code blocks, objects, routines, APIs,functions, etc. The terms “module,” “program,” and “engine” mayencompass individual or groups of executable files, data files,libraries, drivers, scripts, database records, etc.

It will be appreciated that a “service”, as used herein, is anapplication program executable across multiple user sessions. A servicemay be available to one or more system components, programs, and/orother services. In some implementations, a service may run on one ormore server-computing devices.

When included, display subsystem 806 may be used to present a visualrepresentation of data held by storage machine 804. This visualrepresentation may take the form of a graphical user interface (GUI). Asthe herein described methods and processes change the data held by thestorage machine, and thus transform the state of the storage machine,the state of display subsystem 806 may likewise be transformed tovisually represent changes in the underlying data. Display subsystem 806may include one or more display devices utilizing virtually any type oftechnology. Such display devices may be combined with logic machine 802and/or storage machine 804 in a shared enclosure, or such displaydevices may be peripheral display devices.

When included, input subsystem 808 may comprise or interface with one ormore user-input devices such as a keyboard, mouse, touch screen, or gamecontroller. In some embodiments, the input subsystem may comprise orinterface with selected natural user input (NUI) componentry. Suchcomponentry may be integrated or peripheral, and the transduction and/orprocessing of input actions may be handled on- or off-board. Example NUIcomponentry may include a microphone for speech and/or voicerecognition; an infrared, color, stereoscopic, and/or depth camera formachine vision and/or gesture recognition; a head tracker, eye tracker,accelerometer, and/or gyroscope for motion detection and/or intentrecognition; as well as electric-field sensing componentry for assessingbrain activity.

When included, communication subsystem 810 may be configured tocommunicatively couple computing system 800 with one or more othercomputing devices. Communication subsystem 810 may include wired and/orwireless communication devices compatible with one or more differentcommunication protocols. As non-limiting examples, the communicationsubsystem may be configured for communication via a wireless telephonenetwork, or a wired or wireless local- or wide-area network. In someembodiments, the communication subsystem may allow computing system 800to send and/or receive messages to and/or from other devices via anetwork such as the Internet.

In an example, an image forming system comprises: a spatial lightmodulator (SLM) including a plurality of pixels configured to diffractincident light and cause the diffracted light to exit the SLM, where afirst diffraction order of light exiting the SLM passes through a firstexit pupil and higher diffraction orders of light exiting the SLM passthrough one or more additional exit pupils having different positionsfrom the first exit pupil; control logic operatively coupled to theplurality of pixels, the control logic configured to control each pixelto control its modulation of the light incident on the pixel and causethe plurality of pixels to collectively form an image at each exitpupil; a light source configured to emit incident light toward the SLM;and a resampling layer disposed between the light source and SLM, theresampling layer configured to subsample each of the plurality of pixelswith two or more discrete samples per pixel to affect an angle of eachdiffraction order of light exiting the SLM, thereby increasing a spacingbetween each exit pupil at which the image is formed. In this example orany other example, the SLM is a liquid crystal on silicon (LCoS)display. In this example or any other example, the spacing between eachexit pupil at which the image is formed is greater than a predeterminedhuman eye pupil width. In this example or any other example, the lightsource is configured to emit incident light toward the SLM at adynamically controllable incidence angle. In this example or any otherexample, the light source includes a steerable micromirror. In thisexample or any other example, dynamically controlling the incidenceangle changes each respective position at which the first exit pupil andone or more additional exit pupils are formed. In this example or anyother example, the image forming system further comprises a second lightsource configured to emit incident light toward the SLM from a differentincidence angle or position from the light source. In this example orany other example, each of the first exit pupil and one or moreadditional exit pupils are formed at respective first positions when thelight source emits incident light toward the LCoS display, and each ofthe first exit pupil and one or more additional exit pupils are formedat respective second positions when the second light source emitsincident light toward the LCoS display. In this example or any otherexample, an intra-pixel spacing between each of the two or more samplesper pixel is substantially the same for each of the plurality of pixels.In this example or any other example, the two or more samples for eachpixel collectively form a regular sample grid, such that a spacingbetween two adjacent samples is substantially the same whether the twoadjacent samples are on a same pixel or adjacent pixels. In this exampleor any other example, the SLM includes a phase modulating layer and ablurring layer, the blurring layer having a higher permittivity than thephase modulating layer, and the blurring layer is configured to smoothphase transitions of a liquid crystal state in the phase modulatinglayer between localized areas associated with the pixels. In thisexample or any other example, the resampling layer is a microlens arrayconfigured to focus incident light at specific portions of each pixel ofthe plurality of pixels. In this example or any other example, theresampling layer is a mask that selectively blocks incident light fromreaching at least a portion of each of the plurality of pixels. In thisexample or any other example, the resampling layer is a surface reliefgrating (SRG) on a waveguide configured to transmit light via totalinternal reflection. In this example or any other example, theresampling layer is a diffractive optical element. In this example orany other example, the resampling layer modifies a geometric phase ofthe incident light.

In an example, an image forming system comprises: a liquid crystal onsilicon (LCoS) display including a plurality of pixels configured todiffract light incident on the pixel and cause the diffracted light toexit the LCoS display, where a first diffraction order of light exitingthe LCoS display passes through a first exit pupil and higherdiffraction orders of light exiting the LCoS display pass through one ormore additional exit pupils having different positions from the firstexit pupil; control logic operatively coupled to the plurality ofpixels, the control logic configured to control each pixel to controlits modulation of the light incident on the pixel and cause theplurality of pixels to collectively form an image at each exit pupil; alight source configured to emit incident light toward the LCoS display;and an optical array element disposed between the light source and LCoSdisplay, the optical array element configured to selectively redirectthe incident light to subsample each of the plurality of pixels with twoor more discrete samples per pixel to affect an angle of eachdiffraction order of light exiting the LCoS display, thereby increasinga spacing between each exit pupil at which the image is formed. In thisexample or any other example, the light source is configured to emitincident light toward the LCoS display at a dynamically controllableincidence angle, and where dynamically controlling the incidence anglechanges each respective position at which the first exit pupil and oneor more additional exit pupils are formed. In this example or any otherexample, the image forming system further comprises a second lightsource configured to emit incident light toward the LCoS display at adifferent incidence angle from the light source, where each of the firstexit pupil and one or more additional exit pupils are formed atrespective first positions when the light source emits incident lighttoward the LCoS display, and each of the first exit pupil and one ormore additional exit pupils are formed at respective second positionswhen the second light source emits incident light toward the LCoSdisplay.

In an example, a head-mounted display device (HMD) comprises: a spatiallight modulator (SLM) including a plurality of pixels configured todiffract incident light and cause the diffracted light to exit the SLM,such that a first diffraction order of light exiting the SLM passesthrough a first exit pupil and higher diffraction orders of lightexiting the SLM pass through one or more additional exit pupils havingdifferent positions from the first exit pupil; control logic operativelycoupled to the plurality of pixels, the control logic configured tocontrol each pixel to control its modulation of the light incident onthe pixel and cause the plurality of pixels to collectively form animage at each exit pupil; a light source configured to emit incidentlight toward the SLM; and a dynamic resampling layer disposed betweenthe light source and SLM, the dynamic resampling layer configured tosubsample each of the plurality of pixels with two or more discretesamples per pixel to affect an angle of each diffraction order of lightexiting the SLM, thereby increasing a spacing between each exit pupil atwhich the image is formed, where the dynamic resampling layer iscontrollable by the control logic to change a per-pixel sample spacingor sample quantity with which the dynamic resampling layer subsampleseach of the plurality of pixels to dynamically affect the angle of eachdiffraction order of light exiting the SLM, thereby affecting thespacing between each exit pupil at which the image is formed.

It will be understood that the configurations and/or approachesdescribed herein are exemplary in nature, and that these specificembodiments or examples are not to be considered in a limiting sense,because numerous variations are possible. The specific routines ormethods described herein may represent one or more of any number ofprocessing strategies. As such, various acts illustrated and/ordescribed may be performed in the sequence illustrated and/or described,in other sequences, in parallel, or omitted. Likewise, the order of theabove-described processes may be changed.

The subject matter of the present disclosure includes all novel andnon-obvious combinations and sub-combinations of the various processes,systems and configurations, and other features, functions, acts, and/orproperties disclosed herein, as well as any and all equivalents thereof.

1. An image forming system, comprising: a spatial light modulator (SLM)including a plurality of pixels configured to diffract incident lightand cause the diffracted light to exit the SLM, such that two or moredifferent diffraction orders of light pass through two or morecorresponding exit pupils having different, non-overlapping positionswithin an eyebox, at least one of the non-overlapping positions of thetwo or more exit pupils differing from a current eye position of a humaneye relative to the eyebox; control logic operatively coupled to theplurality of pixels, the control logic configured to control theplurality of pixels to collectively form an image at each exit pupil,the image being viewable by the human eye at any of the two or more exitpupils; and a light source configured to emit incident light toward theSLM.
 2. The image forming system of claim 1, where the SLM is a liquidcrystal on silicon (LCoS) display.
 3. The image forming system of claim1, where a spacing between each exit pupil at which the image is formedis greater than a predetermined human eye pupil width, such that onlyone exit pupil of the two or more exit pupils is formed within the humaneye.
 4. The image forming system of claim 1, where the light source isconfigured to emit incident light toward the SLM at a dynamicallycontrollable incidence angle.
 5. The image forming system of claim 4,where the light source includes a steerable micromirror.
 6. The imageforming system of claim 4, where dynamically controlling the incidenceangle changes each respective position at which the two or more exitpupils are formed.
 7. The image forming system of claim 1, furthercomprising a second light source configured to emit incident lighttoward the SLM from a different incidence angle or position from thelight source.
 8. The image forming system of claim 7, where each of thetwo or more exit pupils are formed at respective first positions whenthe light source emits incident light toward the LCoS display, and eachof the two or more exit pupils are formed at respective second positionswhen the second light source emits incident light toward the LCoSdisplay.
 9. The image forming system of claim 1, further comprising aresampling layer disposed between the light source and the SLM, theresampling layer configured to subsample each of the plurality of pixelsto affect a spacing between the different, non-overlapping positions ofeach exit pupil at which the image is formed within the eyebox.
 10. Theimage forming system of claim 9, where an intra-pixel spacing betweeneach of the two or more spatially different sampling locations issubstantially the same for each of the plurality of pixels.
 11. Theimage forming system of claim 9, where the resampling layer is amicrolens array configured to focus incident light at specific portionsof each pixel of the plurality of pixels.
 12. The image forming systemof claim 9, where the resampling layer is a mask that selectively blocksincident light from reaching at least a portion of each of the pluralityof pixels.
 13. The image forming system of claim 9, where the resamplinglayer is a surface relief grating (SRG) on a waveguide configured totransmit light via total internal reflection.
 14. The image formingsystem of claim 9, where the resampling layer is a diffractive opticalelement.
 15. The image forming system of claim 9, where the resamplinglayer modifies a geometric phase of the incident light.
 16. The imageforming system of claim 1, where the SLM includes a phase modulatinglayer and a blurring layer, the blurring layer having a higherpermittivity than the phase modulating layer, and the blurring layer isconfigured to smooth phase transitions of a liquid crystal state in thephase modulating layer between localized areas associated with thepixels.
 17. An image forming system, comprising: a liquid crystal onsilicon (LCoS) display including a plurality of pixels configured todiffract light incident on the pixel and cause the diffracted light toexit the LCoS display, where two or more diffraction orders of lightpass through two or more exit pupils having different, non-overlappingpositions within an eyebox, at least one of the non-overlappingpositions of the two or more exit pupils differing from a current eyeposition of a human eye relative to the eyebox; control logicoperatively coupled to the plurality of pixels, the control logicconfigured to control the plurality of pixels to collectively form animage at each exit pupil, the image being viewable by the human eye atany of the two or more exit pupils; and a light source configured toemit incident light toward the LCoS display.
 18. The image formingsystem of claim 17, where the light source is configured to emitincident light toward the LCoS display at a dynamically controllableincidence angle, and where dynamically controlling the incidence anglechanges each respective position at which the two or more exit pupilsare formed.
 19. The image forming system of claim 17, further comprisinga second light source configured to emit incident light toward the LCoSdisplay at a different incidence angle from the light source, where eachof the two or more exit pupils are formed at respective first positionswhen the light source emits incident light toward the LCoS display, andeach of the two or more exit pupils are formed at respective secondpositions when the second light source emits incident light toward theLCoS display.
 20. A head-mounted display device (HMD), comprising: aspatial light modulator (SLM) including a plurality of pixels configuredto diffract incident light and cause the diffracted light to exit theSLM, such that two or more diffraction orders of light exiting the SLMpass through two or more exit pupils having different, non-overlappingpositions within an eyebox, at least one of the non-overlappingpositions of the two or more exit pupils differing from a current eyeposition of a human eye relative to the eyebox; control logicoperatively coupled to the plurality of pixels, the control logicconfigured to control the plurality of pixels to collectively form animage at each exit pupil, the image being viewable by the human eye atany of the two or more exit pupils; a light source configured to emitincident light toward the SLM; and a dynamic resampling layer disposedbetween the light source and SLM, the dynamic resampling layerconfigured to subsample each of the plurality of pixels to affect anangle of each diffraction order of light exiting the SLM.